I know there are three retention policies for Java annotations:

CLASS: Annotations are to be recorded in the class file by the compiler but need not be retained by the VM at run time.

RUNTIME: Annotations are to be recorded in the class file by the compiler and retained by the VM at run time, so they may be read reflectively.

SOURCE: Annotations are to be discarded by the compiler.

And although I understand their usage scenarios, I don't get why it is such an important thing to specify the retention policy that retention policies exist at all.

I mean, why aren't all the annotations just kept at runtime? Do they generate so much bytecode / occupy so much memory that stripping those undeclared as RUNTIME does make that much difference?